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1. (currently amended) A method of multiattribute analysis and optimization for 
providing automated measurements of an importance of attributes and attribute values of 
message alternatives and, in an automated manner, for improving a stage-to- stage performance 
of the message alternatives in a multi-stage message campaign in an interactive measurable 
medium; the method comprising: 

obtaining historical information including historical performance data for message 
performance for at least one previous stage of the multi-stage message campaign; 

generating a multiattribute system that describes the message alternatives in terms of the 
attributes and the attribute values, wherein an attribute is an element inherent characteristic of a 
message alternative and an attribute value is a particular instantiation of the attribute; 

based on the historical information, determining an importance of the attributes and the 
attribute values to the performance of the message alternatives; 

based on the determination of the importance of the attributes and the attribute values, 
optimizing campaign performance through allocation of message alternatives to visitors during 
the multi-stage message campaign; and 

determining a next allocation for each new visitor for a next stage of the campaign. 

2. (original) A method as in claim 1, further comprising: processing said historical 
performance data to identify erroneous or possibly erroneous or unusual historical performance 
data. 

3. (original) A method as in claim 2, wherein said processing further including 
cleaning said historical performance data to correct said identified erroneous or possible 
erroneous or unusual historical performance data. 
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4. (original) A method as in claim 3, further comprising discounting at least a 
portion of said historical performance data from said at least one previous stage to grant more 
weight to more recent historical performance data than to less recent historical performance data. 

5. (original) A method as in claim 1, further comprising discounting at least a 
portion of said historical performance data from said at least one previous stage to grant more 
weight to more recent historical performance data than to less recent historical performance data. 

6. (original) A method as in claim 5, wherein said historical information further 
includes historical information selected from the set consisting of a visitor identifier, a message 
altemative identifier for the message alternative shown to said visitor, a type of action or types of 
actions taken by each said visitor, and a payoff for each of the actions taken by said visitors. 

7. (original) A method as in claim 1, wherein said historical information further 
includes historical information selected from the set consisting of a visitor identifier, a message 
altemative identifier for the message alternative shown to said visitor, a type of action or actions 
taken by each said visitor, and a payoff for each of the action or actions taken by said visitors. 

8. (original) A method as in claim 6, wherein said historical information further 
includes additional information selected from the set consisting of: a message type, a message 
type and a corresponding alternative clicked on by a visitor, a web site and a section of the web 
site where the visitor was presented with and saw or had an opportunity to see a banner ad, a 
time of the visitors visit, a visitor demographic information, a visitor psychographic information, 
a visitor demographic profile, a visitor psychographic profile, and combinations thereof. 

9. (original) A method as in claim 7, wherein said historical information further 
includes additional information selected from the set consisting of: a message type, a message 
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type and a corresponding alternative clicked on by a visitor, a web site and a section of the web 
site where the visitor was presented with and saw or had an opportunity to see a banner ad, a 

time of the visitors visit, a visitor demographic information, a visitor psychographic information, 
a visitor demographic profile, a visitor psychographic profile, and combinations thereof. 

10. (original) A method as in claim 1, wherein said historical information includes 
additional information selected from the set consisting of: a message type, a message type and a 
corresponding alternative clicked on by a visitor, a web site and a section of the web site where 
the visitor was presented with and saw or had an opportunity to see a banner ad, a time of the 
visitors visit, a visitor demographic information, a visitor psychographic information, a visitor 
demographic profile, a visitor psychographic profile, and combinations thereof. 

1 1 . (original) A method as in claim 1, further comprising: 
processing said historical data to identify and correct any erroneous data; 

discounting at least a portion of said performance data from said at least one previous 
stage to grant more weight to more recent performance data than to less recent performance data; 
and 

said historical information further includes: information selected from the set consisting 
of: a visitor identifier, a message alternative identifier for the message alternative shown to said 
visitor, a t5^e of action or actions taken by said visitor, and a payoff for each of the actions taken 

by said visitors, a message type, a message type and a corresponding alternative clicked on by a 
visitor, a web site and a section where the visitor was presented and saw a banner ad, a time of 
the visitors visit, a visitor demographic information, a visitor psychographic information, a 
visitor demographic profile, a visitor psychographic profile, and combinations thereof. 

12. (original) A method as in claim 11, wherein said message alternative is selected 
from a group of message alternatives consisting of: an email, an ad, a banner ad, a banner, a 
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splash page, a home page, a jump page, a landing page, media programming, media content, a 
political campaign message, a survey, a poll, a news headline, a headline, a ballot measure, a 
ballot initiative, one or more sports scores, one or more sports scores for a local, regional, 
collegiate, or amateur sports team or teams, and combinations thereof. 

13. (original) A method as in claim 1, wherein said message alternative is selected 
from a group of message alternatives consisting of: an email, an ad, a banner ad, a splash page, a 
home page, a jump page, a landing page, media programming, media content, a political 
campaign message, a survey, a poll, a news headline, a headline, a ballot measure, a ballot 
initiative, one or more sports scores, one or more sports scores for a local, regional, collegiate, or 
amateur sports team or teams, and combinations thereof. 

14. (original) A method as in claim 1, wherein each exposure of a message alternative 
to a visitor results in a payoff result, a performance of said message alternative for a visitor 
segment being based at least in part on the expected value of the payoff results for the visitors for 
that visitor segment, said payoff result being a function of a measurement including at least one 
measurement selected from the set of measurements consisting of: a number of actions, a number 
of signups, a number of purchases, a binary result, a value of purchases, a revenue amount, a 
sales amount, a profit amount, a continuous function of a continuous variable, a continuous 
function of a discontinuous variable, a continuous function of a binary variable, a discrete 
function of a continuous variable, a discrete function of a discontinuous variable, a discrete 
function of a binary variable, and combinations thereof. 

15. (original) A method as in claim 12, wherein each exposure of a message 
alternative to a visitor results in a payoff result, a performance of said message alternative for a 
visitor segment being based at least in part on the expected value of the payoff results for the 

visitors for that visitor segment, said payoff result being a function of a measurement including 
at least one measurement selected from the set of measurements consisting of: a number of 
actions, a number of signups, a number of purchases, a binary result, a value of purchases, a 
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revenue amount, a sales amount, a profit amount, a continuous function of a continuous variable, 
a continuous function of a discontinuous variable, a continuous function of a binary variable, a 

discrete function of a continuous variable, a discrete function of a discontinuous variable, a 
discrete function of a binary variable, and combinations thereof. 

16. (previously presented) A method as in claim 14, wherein said multiattribute 
method comprises: 

generating at least one report on an optimal value for each attribute; 

generating new message alternatives with the values of each attribute indicated by a 
multiattribute analysis that uses prior expectations and payoff criteria to arrive at message 
alternative evaluation; 

adding said new message alternatives to the set of available messages to be optimized; 

and 

choosing the best message alternative from among the choices presented and proactively 
creating message alternatives that are expected to perform well within some confidence level. 

17. (original) A method as in claim 16, wherein said multiattribute system is of 
standard form. 

18. (original) A method as in claim 16, wherein said multiattribute system is reduced 
to standard form. 

19. (original) A method as in claim 16, wherein said processing includes discounting 
said performance data; and said discounting being achieved using at least one of discounting 
scheme selected from the set of discounting schemes consisting of: a geometric discounting 
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scheme, a linear weighting discounting scheme, a non-linear weighting discounting scheme, and 
combinations thereof. 

20. (original) A method as in claim 1, wherein said processing includes discounting 
said performance data; and said discounting being achieved using at least one of discounting 
scheme selected from the set consisting of: a geometric discounting scheme, a linear weighting 
discounting scheme, a non-linear weighting discounting scheme, and combinations thereof. 

21. (original) A method as in claim 19, wherein said processing includes estimating 
the distribution of the expected payoff of future performance of said message alternatives. 

22. (original) A method as in claim 21, wherein said estimate is obtained under the 
assumption that the underlying performance model implied by said multiattribute system holds. 

23. (original) A method as in claim 21, wherein said estimate is obtained by first 
estimating the distribution of expected payoff of future performance of said message alternatives 
assuming said underlying performance model implied by said multiattribute system holds, then 
updating the second said estimate to remove said assumption. 

24. (original) A method as in claim 21, wherein said estimate is employed for the 
purpose of allocating future campaign impressions amongst said message alternatives to 
optimize future expected campaign performance. 

25. (original) A method as in claim 19, wherein said allocation method comprises 
performing a pairwise comparison procedure among said plurality of message alternatives. 
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26. (original) A method as in claim 25. wherein said pairwise comparison comprises: 

computing an expected value of a payoff distribution of each message alternative and an 
error in said payoff distribution expected value; and 

comparing in pairwise manner the expected value based statistic of a selected message 
alternative with the same expected value based statistic for each other of the message 
alternatives. 

27. (original) A method as in claim 26, wherein said pairwise comparison comprises: 

selecting a first message alternative and a second message alternative from among said 
plurality of message alternatives; 

computing a quotient determined as difference between the estimate of the said expected 
value divided by a standard error in the difference between the estimate of the expected value for 
each of said first and second message alternatives; and 

determining if said difference is within a predetermined magnitude relationship of, 
including greater than, a threshold cutoff value ci, and: (i) if said difference is within said 
predetermined magnitude relationship of said threshold cutoff value, then identifying the second 
selected message alternative as a member of a non-contending message class. 

28. (previously presented) A method as in claim 27, wherein each message alternative 
is given content, the method further comprising: 

repeating said selecting, said computing a quotient, said computing a difference, and said 
determining for other all other pairs of said message alternatives on a pairwise basis so that when 
all possible pairwise comparisons have been made each message alternative will have been 
sorted into one contender and said non-contender classes. 
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29. (original) A method as in claim 28, further comprising performing said repeating 
for all other pairs of message alternatives on a pairwise basis for each optimization. 

30. (original) A method as in claim 28, wherein said pairwise comparison procedure 
further comprises initially uniformly allocating said message altematives prior to performing a 
first optimization. 

31. (previously presented) A method as in claim 27, wherein the message altematives 
are sorted into a contender class and a non-contender class, and for at least one identified stage in 
a messaging campaign all visitors are allocated equally among message altematives belonging to 
the contender class, and no visitors are allocated to message altematives belonging to the non- 
contender class. 

32. (original) A method as in claim 31, wherein said at least one identified stage 
comprises an initial stage. 

33. (original) A method as in claim 31, wherein said at least one identified stage 
comprises a plurality of stages. 

34. (original) A method as in claim 31, wherein said at least one identified stage 
comprises all stages. 

35. (previously presented) A method as in claim 25, wherein said allocation 
comprises a pairwise comparison, and message altematives are sorted into more than two 
classes. 
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36. (original) A method as in claim 35, wherein said messages belonging to each said 
class are assigned different numbers of visitors. 

37. (original) A method as in claim 35, wherein said messages belonging to each said 
class are assigned substantially the same numbers of visitors. 

38. (previously presented) A method as in claim 25, wherein said allocation 
comprises a pairwise comparison, and message alternatives are sorted into more than two classes 
and the allocation among contenders is not equal. 

39. (previously presented) A method as in claim 25, wherein said allocation 
comprises a pairwise comparison, and message alternatives are sorted into more than two classes 
and the allocation among non-contenders is not equal. 

40. (previously presented) A method as in claim 25, wherein said allocation 
comprises a pairwise comparison, and message alternatives are sorted into more than two classes 
and the allocation among contenders is not equal and the allocation among non-contenders is not 
equal. 

41. (previously presented) A method as in claim 27, wherein said allocation 
comprises a pairwise comparison, and the cutoff Ci is in the range substantially between 0-1.0. 

42. (previously presented) A method as in claim 27, wherein said allocation 
comprises a pairwise comparison, and the cutoff ci is substantially 0.5. 
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43. (previously presented) A method as in claim 27, wherein said allocation 
comprises a pairwise comparison, and the cutoff ci is greater than or equal to 1. 

44. (previously presented) A method as in claim 27, wherein said allocation 
comprises a pairwise comparison, and the cutoff ci is greater than 1. 

45. (original) A method as in claim 16, wherein reports on the optimal value for each 
attribute are provided only if it is determined that said multiattribute system supports said 
measurement of the relative importance to said message alternative performance of said 
attributes and said attribute values. 

46. (original) A method as in claim 45, wherein said multiattribute system is reduced 
to standard form to determine if said multiattribute system supports said measurement of the 
relative importance to said message alternative performance of said attributes and said attribute 
values. 

47. (original) A method as in claim 45, wherein determination of said relative 
importance to said message alternative performance of said attributes and said attribute values 
includes determination of confidence intervals around said relative importance. 

48. (previously presented) A method as in claim 16, wherein reports on the optimal 
value for each attribute are provided upon a given occurrence. 

49. (previously presented) A method as in claim 48, wherein an acceptance/rejection 
test is employed upon a given occurrence. 
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50. (original) A method as in claim 48, wherein reports on said relative importance to 
said message alternative performance of said attributes and said attribute values are provided to 
the marketing manager or other interested party. 

51. (original) A method as in claim 48, wherein determination of said relative 
importance to said message alternative performance of said attributes and said attribute values 
includes determination of confidence intervals around said relative importance. 

52. (previously presented) A method as in claim 50, wherein said performance 
measurement represents click-through rates of the message alternatives, the multiattribute 
function is a logodds function, and the reported relative importance to the message alternative 
performance of one of the attribute values is computed as 100*(exp{M}-l), where u represents a 
parameter estimate of a multiattribute parameter associated with the attribute value, and the 
reported relative importance to the message alternative performance of one of the attributes is 
computed as 100x(exp{M}-l), where u represents the largest parameter estimate of a 
multiattribute parameter associated with the attribute values of the attribute. 

53. (original) A method as in claim 52, wherein said reports include a chart and a 

table. 

54. (original) A method as in claim 53, wherein said chart is a bar chart where the 
length of the bars are determined by said reported relative importance of said attributes or said 
attribute values. 

55. (original) A method as in claim 54, wherein the colors of said bars are determined 
by assigning each attribute a color, and whenever an attribute or a value of said attribute appears 
on a chart, coloring said bar with said color. 
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56. (original) A method as in claim 54, wherein the colors of said bars are determined 
by, when reporting on an attribute, using one color if the lower confidence bound of the best 
attribute value for said attribute is less than zero, and using a second color otherwise; and 
likewise when reporting on an attribute value, using one color if the lower confidence bound of 
said attribute value is less than zero, and using a second color otherwise. 

57. (original) A method as in claim 54, wherein the colors of said bars are determined 
by assigning each attribute a color and each attribute value a variant of the color assigned to its 
attribute, and whenever an attribute or an attribute value appears on a chart, coloring said bar 
with said color. 

58. (currently amended) A computer program product for use in conjunction with a 
computer system, the computer program product comprising a computer readable storage 
medium and a computer program mechanism embedded therein, the computer program 
mechanism comprising: a program module that directs the computer system and/or components 
thereof, to function in a specified manner to provide automated measurements of an importance 
of attributes and attribute values of message alternatives and, in an automated manner, to 
improve a stage-to-stage performance of the message alternatives in a multi-stage message 
campaign in an interactive measurable medium, the program module including instructions for: 

obtaining historical information including performance data for message performance for 
at least one previous stage of the multi-stage message campaign; 

generating a multiattribute system that describes the message alternatives in terms of the 
attributes and the attribute values , wherein an attribute is an inherent characteristic of a message 
alternative and an attribute value is a particular instantiation of the attribute ; 

based on the historical information, determining an importance of the attributes and the 
attribute values to the performance of the message alternatives; 
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based on the determination of the importance of the attributes and the attribute values, 
optimizing campaign performance through allocation of message alternatives to visitors during 
the multi-stage message campaign; and 

determining a next allocation for each new visitor for a next stage of the campaign. 

59. (original) A computer program product as in claim 58, wherein said program 
module further including instructions for: 

processing said historical data to identify and correct any erroneous data; 

discounting at least a portion of said performance data from said at least one previous 
stage to grant more weight to more recent performance data than to less recent performance data; 
and wherein: 

said historical information further includes additional information selected from the set 
consisting of: a visitor identifier, a message alternative identifier for the message alternative 
shown to said visitor, a type of action or actions taken by said visitor, and a payoff for each of 
the actions taken by said visitors, a message type, a message type and a corresponding alternative 
clicked on by a visitor, a web site and a section where the visitor was presented and saw a banner 
ad, a time of the visitors visit, a visitor demographic information, a visitor psychographic 
information, a visitor demographic profile, a visitor psychographic profile, and combinations 
thereof. 

60. (original) A computer program product as in claim 58, wherein said message 
alternatives are selected from a group consisting of: an email, an ad, a banner ad, a splash page, a 
home page, a jump page, a landing page, media programming, media content, a political 
campaign message, a survey, a poll, a news headline, a headline, a ballot measure, a ballot 
initiative, one or more sports scores, one or more sports scores for a local, regional, collegiate, or 
amateur sports team or teams, and combinations thereof. 
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61-63. (cancelled) 
64-66. (withdrawn) 

67. (new) A method of identifying a relative performance of a set of creatives during 
a multi-stage message campaign, wherein a given creative has associated therewith two or more 
attributes, and each attribute has two or more values, wherein an attribute is an inherent 
characteristic of a message alternative and an attribute value is a particular instantiation of the 
attribute, comprising: 

defining a set of multiattribute data structures and assigning the creatives to the set such 
that each creative is assigned to one and only one multiattribute data structure; 
receiving data indicative of a performance of the creatives; 

using the performance data, estimating values of a set of multiattribute parameters for 
each of the muUiattribute data structures, wherein the values of the set of multiattribute 
parameters define relative impact of the attribute values on performance of the creatives; and 

using the values, optimizing campaign performance through allocation of the set of 
creatives during a given stage of the multi-stage message campaign; 

wherein at least one or more of the steps are performed by one or more electronic 
processing devices. 

68. (new) The method as described in claim 67 further including the step of reducing 
the set of multiattribute data structures to a standard form prior to the estimating step. 

69. (new) The method as described in claim 67 further including the step of 
processing the performance data by a given discounting function prior to the estimating step. 
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